TRUNCATED TOEPLITZ OPERATORS AND BOUNDARY VALUES IN NEARLY 

INVARIANT SUBSPACES 



ANDREAS HARTMANN & WILLIAM T. ROSS 

Abstract. We consider truncated Toeplitz operator on nearly invariant subspaces of the Hardy 
space H^. Of some importance in this context is the boundary behavior of the functions in these 
spaces which we will discuss in some detail. 



1. Introduction 

If H'^ is the classical Hardy space HDurVOL we say a (closed) subspace M c H'^ is nearly 
invariant when 

/€M,/(0) = 0^^6M. 

z 

These subspaces have been completely characterized in [|Hit88[|Sar88ll and continued to be stud- 
ied in IIAR96[ IHSS041 IMPOSi IKN06[ IAK08[ ICCPIOL In this paper, we will examine certain 
properties of nearly invariant subspaces. We aim to accomplish three things. 

First, in Theorem 13. II we observe that much of what we already know about truncated Toeplitz 
operators on model spaces [SarOV] can be transferred, via a unitary operator to be introduced 
below, mutatis mutandis to truncated operators on nearly invariant subspaces. 

Secondly, in Theorem 14.51 we will show that every function in a nearly invariant subspace M 
has a finite non-tangential limit at |C| = 1 if and only if (i) g has a finite non-tangential limit at 
C and (ii) the reproducing kernel functions for M are uniformly norm bounded in Stolz regions 
with vertex at C,. This parallels a result by Ahem and Clark in model spaces. 

Third, in order to better understand the self-adjoint rank-one truncated Toeplitz operators on 
nearly invariant subspaces, we will discuss the non-tangential limits of functions in these spaces. 
It turns out (Theorem 15.21) there is a kind of dichotomy: If every function in a nearly invariant 
subspace M has a boundary limit at a fixed point \C\ - 1 then either ^ is a point where every 
function in (where g is the extremal function for M - see definition below) has a finite 
non-tangential limit or every function in M has non-tangential limit at 
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A word concerning numbering in this paper: in each section, we have numbered theorems, 
propositions, lemmas, corollaries and equations consecutively. 

2. Preliminaries 

If H"^ is the Hardy space of the open unit disk D (with the usual norm || • ||) and / is inner, 
let Ki = e IH"^ be the well-studied model space llNik86ll . Note that i/^, as well as Kj, are 
regarded, via non-tangential boundary values on T := dlD), as subspaces of := L'^{T,d9/2Tc) 
HDurVOH . It is well known that H"^ is a reproducing kernel Hilbert space with kernel 

1- Xz 

as is Kj with kernel 

^ l-Xz 

Note that /cj^ is bounded and that finite linear combinations of them form a dense subset of Kj. 
Also note that if Pj is the orthogonal projection of onto Kj, then 

{Pjf){X) = {f,ki), XeB. 

Note that for every A e D this formula extends to / e = L^{T,d9/2n) so that we can define 
Pj for L^-functions (which in general does not give an i/^-function). 

By a theorem of Ahem and Clark IIAC70II . every function / e has a finite non-tangential 
limit at a boundary point C ^ T, i.e., 

/(C) :=^lim/(A) 

exists for every / e Kj, if and only if C ^ ADC(/). Here ( e ADC(/) means that / has a finite 
angular derivative in the sense of Caratheodory, meaning 



and 



^lim/(A) =r]eT 



^ lim/'fA) exists. 



Moreover, whenever ( £ ADC(/), the linear functional / >-> f{Q is continuous on Ki giving us 
a kernel function /c^ for Kj at the boundary point (. That is to say 

/(C) = (/,^c) 

Sarason [|Sar07ll began a study, taken up by others, of the truncated Toeplitz operators on Kj. 
These operators are defined as follows: For if e L"^, define the truncated Toeplitz operator 
densely on the bounded / € Kj by 

AJ = Pr{^f). 

Let 7/ denote the which extend to be bounded operators on Ki. Certainly when cp is bounded, 
then A^ e Ti, but there are unbounded which yield A^ € Ti. Moreover, there are bounded 
truncated Toeplitz operators which can not be represented by a bounded symbol liBCF'^lOI I. Much 
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is known about these operators (see the Sarason paper USarOVH for a detailed discussion) but we 
list a few interesting facts below: 

(1) 7/ is a weakly closed linear subspace of operators on Kj. 

(2) E if and only if ipeIH^+ JIP. 

(3) The operator C f ■= zfl (considered as boundary functions on T) defines an isometric, 
anti-linear, involution on Kj for which CA^C = A'^ = Aj^. The operator C is called a 
conjugation. This makes 7/ a collection of complex symmetric operators IIGP061IGP07II . 

(4) A bounded operator A on Kj belongs to 7} if and only if there are functions (pi,^P2 ^ Kj 
so that 

A - A^AAl = {^i ® kl) + {ki ® ip2)- 
In the above, we use the notation f ® gfor the rank-one operator 

U^gm ■.= {h,g)f. 

(5) An operator A on Ki is a rank-one truncated Toeplitz operator on if and only if it can be 
written as constant multiple of one of the following three types 

k{®Ck{, Cki^ki, kl®kl 

where A € ©, C e ADC(/). The two first ones are the non- self adjoint and the last ones 
are the selfadjoint truncated Toeplitz operators. 

(6) Sedlock USedl (see also HGRWH ) showed that every maximal algebra in 7/ can be written 
as the commutant of a generalization of the Clark unitary operator. 

(7) In HCGRWIO I they show, for two inner functions Ji and I2, that Ti^ is spatially isomor- 
phic to 7/2 if and only if either Ji = if) o I2 o cj) ox Ii = ijj o l2{z) o for some disk 
automorphisms (p, ip. 

As we have already mentioned, a nearly invariant subspace M is a closed subspace of H"^ 
such that 

/€M,/(0) = 0^^€M. 

z 

A result of Hitt F HitSSH says that if g (called the extremal function for M) is the unique solution 
to the extremal problem 

(2.1) sup{Re(?(0):(/€M,||^|| = 1}, 

then there is an inner function / so that 

M = gKi 

and moreover, the map 

Ug-.Ki^M, UJ = gf, 

is isometric. 

Note that necessarily we have the condition /(O) = since g € M = gKj and so 1 € Ki. If 
one were to choose any g and any inner /, then gKi is, in general, not even a subspace of 77^. 
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Sarason shows in USarSSH that when /(O) = 0, every isometric multiplier on Ki takes the form 

a 



where a and h are in the unit ball of H°° with |ap + |6p = 1 a.e. on T. As a consequence, gKi is 
a (closed) nearly invariant subspace of H"^ with extremal function g as in (12.11) . 

Remark 2.3. From now on, whenever we speak of a nearly invariant subspace M - gKj, we 
will always assume that 1(0) = and that g is extremal for M (as in (12.11 )) and of the form in 
(12.21 ). We will say that gKj is a nearly invariant subspace with extremal function g and suitable 
inner function / with /(O) = 0. 

Our first step towards defining a truncated Toeplitz operator on the nearly invariant subspace 
M - gKj, as Sarason did for Kj, is to understand Pm, the orthogonal projection of onto M. 

Lemma 2.4. Let M = gKj be a nearly invariant subspace with extremal function g and associ- 
ated inner function I, /(O) = 0. Then 

PMf = gPiigf), f^M. 

Proof. To show that the map / gPi{gf) is indeed Pm, we need to show that this map is the 
identity on M and vanishes on e M. 

Let f = ghe gKj with h € Kj. Then for every A e D we have 

iPiggh)iX) = {ggh,ki) = {gh,gk{). 

But, from our previous discussion, multiplication by the extremal function g is an isometry from 
Kj onto M and so 

{gKgki) = {h,ki) = h{X). 

Thus 

^(A)(P7^/)(A) = /(A), 
in other words, / gPi{gf) is the identity on M. 

To finish the proof, we need to show that the map / >^ gPi{gf) vanishes on e M. Clearly, 
when f eHl = {h:he H^, h{0) = 0} then Pi{gf) = 0. When f e e M we have 

{Pigf){X) = {gf,ki) = {f,gki) = 

since gk{ e M and f 1 M. ■ 

Corollary 2.5. The reproducing kernel for M = gKj is given by 



kt!{z)=g{\)g{z) 



l-/(A)/(z) 



l-Xz 

Proof Observe that k^' {z) = {PMk\){z) = g(z){Pigkx){z) and 



iPigk,){z) = {gkx,ki) = {k,,gki) = (gkDiX) = g{X)k{{z). 
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3. Truncated Toeplitz operators on nearly invariant subspaces 

We are now able to introduce truncated Toeplitz operators on nearly invariant subspaces. Cer- 
tainly whenever (/? is a bounded function we can use Lemma [Z4l to see that the operator 

A'Jf:=PM^f = gPi{g^f), feM, 

is well-defined and bounded. 

The most general situation is when the symbol ip is a Lebesgue measurable function on T 
and \g\'^(p £ L^. Then, for every bounded h e Ki, the function \g\'^iph belongs to and so 
Pi{\g\'^iph) € Kj, and, by the isometric multiplier property of g in Kj, we get 

PM{^h)=gPj(\g\'^h)egKi = M. 

Note that by the isometric property of g, the set gK^, where Kf" is the set of bounded functions 
in Ki, is dense in gKj. Hence, in this situation, is densely defined. 

Let T^^ denote the densely defined A^^ , \g\'^ip £ L^, which extend to be bounded on M and 
recall that Ug : Kj M, Ugf = gf, is unitary. 

Theorem 3.1. Let M = gKj be a nearly invariant subspace with extremal function g and suitable 
inner function I, 1(0) = 0. Then for any Lebesgue measurable function (p onT with \g\'^(p e L'^ 
we have 

U*gA^'Ug = A\g\2^. 

Proof. Let he Kj, then 

A^'Ugh = A^'gh = gPi\g\^p>h = UgA^gp^h. 



The above shows that the map 

establishes a spatial isomorphism between T*^ (the bounded truncated Toeplitz operators on M) 
and 7} (the bounded truncated Toeplitz operators on Kj) induced, via conjugation, by the unitary 
operator Ug:Ki^M = gKj, i.e., T^^ = UgTiU*. 

In view of the results in Sarason's paper USarOVII one can use the above spatial isomorphism to 
prove the following facts: 

(1) 7m is a weakly closed linear subspace of operators on M. 

(2) E if and only if \g\^^ e IH^ + HP. 

(3) Recalling the conjugation C mentioned earlier, define Cg ■■= UgCU*. This defines a 
conjugation on M and CgACg = A* for every A e T^'^, and since (Aj^)* = A^' (as 
can be verified by direct inspection), we see that T^^ is also a collection of complex 
symmetric operators (with respect to Cg). 
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(4) If Sg ■= UgAzUg, then a bounded operator Aon M belongs to T^^ if and only if there are 
functions ipi,ip2 ^ M so that 

A - SgAs; = {if, ® C) + (C ® ^2). 

(5) Recall from our earlier discussion of USarOVII that A is a rank-one truncated Toeplitz 
operator on Kj if and only if it can be written as constant multiple of one of the following 
three types (two of which are non-selfadjoint, and one of which is selfadjoint) 

ki ® Cki, Ck{ ® ki, kl kl 

where A € D, ( € ADC(/). It follows immediately that for A e ©, 

Ug{ki Cki)U; = Ugki ® UgCki = Q k'^ ® gCkl 
UgiCki ki)U; = UgCki Ugki = gCki ^ fc^ , 

are rank-one truncated Toeplitz operators on M = gKi. What are the self-adjoint trun- 
cated Toeplitz operators on Ml The result here is the following: Suppose that C, e 
ADC(/), then 

gkl gkl 

is a self-adjoint rank-one truncated Toeplitz operator on M. Conversely, if A is a non- 
trivial rank-one self adjoint truncated Toeplitz operator on M, then there exists a e 
ADC(/) so that A = gk^^® gkj.. 

(6) Recall from our earlier discussion that every maximal algebra in 7/ can be written as the 
commutant of a generalization of the Clark unitary operator. Conjugating by Ug we get 
an analogous result for the maximal algebras of 7"^^. 

(7) Again from our earlier discussion, for two inner functions Ii and I2, that Ti^ is spatially 
isomorphic to 7/2 if and only if either Ji = ^ o o or Ji = ^ o l2{z) o for some disk 
automorphisms (j),^p. Using Theorem 13. II we get the exact same result for 7~a/i 
Notice how this result is independent of the corresponding extremal functions gi and g2. 

4. Existence of non-tangential limits 

With trivial examples one can show that there is no point ^ e T such that every / e H"^ has a 
non-tangential limit at (. However, there are model spaces Kj and ^ £ T so that every f ^ Kj has 
a non-tangential limit at (. This situation was thoroughly discussed by Ahem and Clark [IAC70II 
with the following theorem. For C eT and a > 1 let 

r.(C) :=|^€©:t_^|<«| 
be the standard Stolz domains with vertex at 

Theorem 4.1 (Ahern-Clark). Let I be an inner function. Every function f e Kj has a non- 
tangential boundary limit at C e T if and only if for every fixed Stolz domain Va{C,) at (, the 
family 

F,,C :={fci:A€r,(C)} 
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is uniformly norm bounded, i.e., 

sup|||fc{P = ^^^:AeUC)|<oo. 

The original proof of this theorem involved a technical lemma using operator theory. We will 
give a new proof which avoids this technical lemma. This will allow us to consider the situation 
of nearly invariant subspaces where the operator theory lemma of Ahem-Clark no longer works. 
Also note that the uniform boundedness of the families F^,^ is equivalent to the condition C e 
ADC(/). 



Proof of Theorem \4J] From the uniform boundedness principle it is clear that if every function 
in Kj has a non-tangential limit at e T, then the kernels kj^ are uniformly bounded in Stolz 
domains ra(C). 

Let us consider the sufficiency part. If, for fixed a and C, the family is uniformly bounded, 
then there exists a sequence A := {A„}„>i c r„(C) such that converges weakly to some 
fc^'^ € Kj, i.e., for every function / € Kj, 

(4.2) f{K) = {f,kiJ^{f,kl''')=:f^ asn-oo. 

Observe that 



and 



XN 

where /(j) is the j-th Fourier coefficient of /. As a result we have 

oo 

II^#/IP= El/O')l'-0 asiV-oo. 

j=N 

Apply (1431) to the functions f to get 
Hence 



{A^fX^^) = lim(A#/,fcl)= lim 
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Now using the fact that \\A^f\\-^0 when ^ oo, we obtain from {A^ /, k') that 



C 



lim ^ = 0, 

and, since the denominator in the above limit is harmless, we get 

oo 
j=0 

This means that the Fourier series for / converges at (. 

Now take any sequence A := {A„}„>i converging non-tangentially to In view of the uniform 
boundedness of the family F^,^^, there exists a weakly convergent subsequence A' := {kj^^ }/>i. 
Repeating the above proof, we obtain 

oo 

j=0 

In other words, for every sequence {A„}„>i converging non-tangentially to ( there exists a sub- 
sequence {A„^ }j>i such that {/(A„^ )}j>i converges to the same limit 

oo 

3=0 

It follows that / has a finite non-tangential limit at (. ■ 

We will now prove an analog of Theorem 14.11 for nearly invariant subspaces where the sit- 
uation is a bit more complicated. Certainly a necessary condition for the existence of finite 
non-tangential limits for every function in gKj at ( is that g has a finite non-tangential limit at 
( (since indeed g is extremal and so g e gKj). As we will see with the next example, it is not 
possible to deduce the existence of the non-tangential limits for every function in gKi merely 
from the uniform norm boundedness of the reproducing kernels in a Stolz angles. 

Example 4.3. Given any inner function /, with /(O) = 0, we have already cited Sarason's result 
ffSarSSJ stating that every isometric multiplier on Kj is of the form 

(4.4) 9 = ^—, 

where a and b are functions in the ball of H°° with |ap + |6p = 1 a.e. on T. Let Ai = {1 - 4""}„>i 
and A2 = {1 - 2 "}„>i, which are both interpolating sequences. Observe that A2 consists of the 
sequence Ai to which we have added in a certain way the (pseudohyperbolic) midpoints of two 
consecutive points of Ai. Let Bi be the Blaschke product whose zeros are Aj. In particular, Bi 
vanishes on Ai and is big on A2 \ Ai, say \Bi (A)| > S > for A e A2 \ Ai (this comes from the fact 
that Bi is an interpolating Blaschke product, and that on A2 \ Ai we are pseudohyperbolically 
far from the zeros of Bi). 

Next, define 

a = - + -Bi 

2 4 
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which is a function oscillating on A2 between the values 1/2 (on Ai) and |a(A) - 1/2| > djA, 
A e A2 \ Ai. 

By construction we have 1/4 < \a\ < S/i. In particular there is an outer function in the ball 
of H°° such that |ap + |6op = 1 a.e. on T. Moreover < V7/4 < |6o| = v'l - lap < ^/TE/A < 1 
a.e. on T and this extends by the maximum/minimum principles to the disk (69 being outer). As 
a consequence, for every inner function J, the function g = a/{l- Jbo) will be a bounded outer 
function (actually invertible). We will use this fact in particular for J - IB2: 



1-Jbo I- I B2bo 

=bin{431 



Then 

1 



9(A) -i 



a(A) 




when A e Ai 
when A e A2 \ Ai . 

Hence g has no limit at ^ = 1- Choose now an inner function / such that C = 1 ^ ADC(/). Then, 
the kernels = g{X)gk{ are uniformly bounded in a fixed Stolz region Ta{l) at ( = 1. Indeed 
g is bounded, the kernels kj^ are uniformly bounded in rQ,(l) by 1 e ADC(/) (see Theorem 14 .11) . 

and \\ki'\\ = \gm\kil 

Theorem 4.5. Let M = gKj be a non-trivial nearly invariant subspace with extremal function g 
and corresponding inner function I. Then every function in M has a finite non-tangential limit 
at ( if and only if the following two conditions are satisfied: 



(1) g has a finite non-tangential limit at 



(2) For every Stolz region T^iC) C the family F^l := {k¥ : XeTa{C)} uniformly norm 



bounded, i.e., 



supji^f IP = \g(X)\-ljI^ : A e r.(C)| < 00. 



Proof. If every function in M has a finite non-tangential limit at (, then (1) holds since g e M. 
Condition (2) holds by the uniform boundedness principle. 

Now suppose that conditions (1) and (2) hold. 

Case 1: ( € ADC(/). In this case, every function in M has a non-tangential boundary limit at 
( since every function in Ki has as well as g does. 

Case 2: / ADC(/). From the Ahern-Clark theorem (Theorem 14.11) . we know that ( e 
ADC(/) if and only if 

II^aII = ^_|;^|2 

is uniformly bounded in Stolz regions ra(C)- Hence, if C ^ ADC(/), then there is a sequence 
A := {A„}„>i in TaiO such that \\k{ P ^ +00. Recall that kj^^ = g{\)gk{ so that jk^ = 
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II II . In order that this last expression is uniformly bounded it is necessary that g{Xn) 
when n ^ oo. 

?M 



Now, since F^^^ is a uniformly bounded family, there exists a subsequence A' := {A„^}j>i so 
it k¥ converges weakly to a function kf^'^ . 

Let us introduce the following operator 



QN = UgA^U; 

from M = gKi to itself. Let f = gh€ gKj = M. Recall that U*f = h. Then 

^iV-l- 



(QNf){z) = ({U,A^U;)f){z) = [{U,A^hyz)=giz) 



Hz) -TZZo h{m)z^^ 
z^ 



f{z) g{z)'i-}-r. . 

—N Zn- L H^)' 



Z' 



m=0 



Now since Qn/ ^ M, we get, from the weak convergence of k¥ kf'^'^ as j ^ oo 



An, "'C 



(4.6) {Q^f, k^''^') = lim(g^/)(A„^,) = lim - j:h{m)X 



^rij m=0 



The last sum appearing above is a harmless polynomial converging to some number when j ^ oo. 
Also A„ C and g{\n ) -^0 when j ^ oo. Hence the limit in (14.61 ) is equal to 



where 



fA' 



M,A'\ _ ■'C 



which exists by construction. 
Since 



llQ^/i = \\U,A^h\\ = \\gA^h\\ = \\A^h\\ - as - oo, 



we have 

which is possible precisely when f^' = 0. 

The above construction is independent of the choice of the sequence {A„}„>i so that for every 
such sequence there is a subsequence {A„^}j>i such that /(A„^) when j oo. As in the 
proof of Theorem 14 . 1 1 we conclude that ^ limA^( /(A) exists and is, in fact, equal to 0. 
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5. The dichotomy 



For a nearly invariant subspace M = gKj, let 

AT^ = |C e T : ^ lim/(A) = V/ € m| 



The sets ADC(/) and N^"^ are not necessarily disjoint. If the function g vanishes at a point 
C e ADC(/) then automatically every function / e gKj vanishes at C so that ( e A^^^. 

Another situation is the following. Observing that every function / e H'^ satisfies the following 
well known growth condition 

(5.1) 1/(^)1 = 



we see that whenever \g{z)\ = o{^\z - C|) in a Stolz domain rQ,(C) based on (, every function 
gh tends non tangentially to at (, and so ^ ^ A^^^ independent of /. Notice that for growth 
in a Stolz domain rQ,(C) we can replace "big-Oh" by "little-oh" in (15.11) . and then "little-oh" by 
"big-oh" in the growth of g in r„((^). The main theorem of this section is the following. 

Theorem 5.2. Let M be a non-trivial nearly invariant subspace with extremal function g and 
associated inner function I. Suppose that every function in gKj has a finite non-tangential limit 
atCeT. Then ( e ADC(/) or ( e N^. 

Note again that the intersection ADC(/) n A^^^ can be non-empty. 

Proof of Theorem |572l If every function in gKj has a non-tangential limit at C ^ T, then, in 
particular, g has a non-tangential limit at As a reminder, recall that g belongs to gKj since 
/(O) = (and thus the constants belong to Kj). 

If 

^ Urn (7(A) ^0, 

then, since every function f = gh e gKj has a finite non-tangential limit at Q, we can divide 
by g to get h = f /g e Kj which has a non-tangential limit at (. So, every function in Kj has 
non-tangential limit and we conclude from [iAC70il that e ADC(/). 

Now suppose that 
(5.3) ^\img{X) = 0. 

First note that if every function in gKj has a non-tangential limit at ( e T, then, as already 
mentioned, there exists a reproducing kernel k^'^ e M at Let now {A^}^>i be any sequence 
with A.„ € Ta{0 tending to (. Then for z e]Q), 



kf{z) = {kfX') = {k^',kf) = k^iO = lim (A„) = lim lg(z)g(X^)LJiMM. 

^ ^ ^ n-s-oo n-s-oo \ 1 — X Z 
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Since 2; e D is fixed, by (15.31) . the above limit will be zero, which forces the kernel k^^ to vanish 
identically. Thus for every f e M = gKj we have 

/(C) = lim(/,fc,^^) = (/,0) = 0. 

■ 

As a consequence of this result, if C ^ \ ADC(/), then the point evaluation operator at C 
is just the zero-functional. We state this observation as a separate result. 

Corollary 5.4. Let M be a non-trivial nearly invariant subspace with extremal function g and 
associated inner function I. On M, the only non-zero point evaluation functionals at a point 
C 6 T are those for which ( e ADC(/) and g admits a non-tangential limit different from zero at 

C- 
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